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Introduction 

Let G = PGL 2 (M) and let 7ri,7r2,7r3 be irreducible admissible smooth repre¬ 
sentations of G. The the space of G-invariant trilinear forms on tti x 7r2 x tts 
is at most one dimensional. This has, in different contexts, been proved 
by Loke j3], Molchanov jS] and Oksak [7j. In this paper we ask for such a 
uniqueness result in the context of arbitrary semisimple groups. We show 
that for a given group G uniqueness can only hold if G is locally a product 
of hyperbolic groups. For such groups we show uniqueness and for spherical 
vectors we compute the invariant triple products explicitly. 

The PGL 2 -result has been used in pQ to derive new bounds for automorphic 
L^-coefficients. This can also be done for higher dimensional hyperbolic 
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groups, but, with the exception of the case treated in P, the results do not 
exceed those in p. For completeness we include these computations in an 
appendix. 

I thank J. Hilgert and A. Reznikov for helpful comments on the subject of 
this paper. 


1 Representations and Integral formulae 

Let G be a connected semisimple Lie group with hnite center. Fix a maximal 
compact subgroup K. Let G and K denote their unitary duals, i.e., the sets 
of isomorphism classes of irreducible unitary representations of G resp. K. 
Let TT be a continuous representation of G on a locally convex topological 
vector space 14- Let IG Le the space of all continuous linear forms on 14 and 
let be the space of smooth vectors, i.e., 

14 °° = {v ^ Vtt : X a{'K{x)v) is smooth Va G l4}. 

The representation tt is called smooth if I 4 = 14°°- 

A representation (tt, I 4 ) is called admissible if for each r E K the space 
Hom/f(14,14) is hnite dimensional. Let Gadm be the admissible dual, i.e., 
the set of inhnitesimal isomorphism classes of irreducible admissible repre¬ 
sentations. A representation vr in Gadm is called a class one representation 
if it contains iF-invariant vectors. In that case the space 14^ of iF-invariant 
vectors is one-dimensional. 

The Iwasawa decomposition G = ANK gives smooth maps 

a: G ^ A 
n-.G^N 
k-.G^K 

such that for every x E G one has x = a{x)n{x)k{x). As an abbreviation we 
also dehne gm{x) = a{x)n{x). Let 0 R, Or, Ur, denote the Lie algebras of 
G, A, N, K and let g, 0 , n, fi be their complexihcations. 

For X E G and k E K we dehne 

k^ = k{kx). 
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Lemma 1.1 The rule k ^ defines a smooth (right) group action of G on 

K. 


Proof: The map k gives a diffeomorphism AN\G K and the action under 
consideration is just the natural right action of G on AN\G. □ 

Lemma 1.2 For f G G{K) and y E G we have the integral formula 

[ f{k)dk= [ gfkyfPf{ky)dk, 

Jk Jk 


or, 


f{ky)dk= / a{ky-^fP f{k)dk. 


>K 


>K 


Here p E a* is the modular shift, i.e., off’ = det(a|n). 

Proof: The Iwasawa integral formula implies 

/ g{x)dx= / / g{ank)dkdan. 

Jg Jan Jk 

Let / G G{K) and choose a function rj E GfiAN) such that p > 0 and 
dadn = 1. Let g{x) = ri{gm{x))f{k{x). Then g E GfiG) and 


/ g{x)dx = / r](an) dan / f{k)dk 

'g Jan Jk 


f{k) dk. 


IK 
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On the other hand, since G is unimodular, this also equals 


g{xy)dx = / ri(an(xv)) fikjxy)) dx 


IG 


>G 


nianjanky)) f(k(ky)) dank 


I AN J K 

<' \ 

nianjanky)) dan j /(/c^) dk 

Ik \Jan j 

r \ 

niananjky)) dan 1 fjkL) dk 

IK \JAN J 

[ ( [ yjan) da7i\ fjkL) dk 

Ik \Jan j 

[ QLikyY^f jk^) dk. 

Ik 


The second assertion follows from the hrst by replacing / with fjk) = fjk^) 
and then y with y~^. □ 


The inclusion map K ^ G induces a diffeomorphism M\K P\G and in 
this way we get a smooth G-action on M\K. An inspection shows that this 
action is given by Mk i—> Mk^ for k & K, x & G. 


2 Trilinear products 

Let TTi, 7r2, TTa be three admissible smooth representations of the group G and 
let T: fdrj X K. X fdrg —C be a continuous G-invariant trilinear form, i.e., 

Ti7iiix)vi,n2ix)v2,n3ix)v3) = Tivi,V2,V3) 
for all Vj G and every x E G. 

We want to understand the space of all trilinear forms T as above. Since any 
irreducible representation is a quotient as well as a sub-representation of a 
principal series representation it suffices to assume that the tij are principal 
series representations. This means that there are given a minimal parabolic 
P = MAN, irrudicible representations aj G M, and Xj G o* for j = 1, 2, 3. 
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Each pair (aj, Xj) induces a continuous group homomorphism P GL(t4^) 
by man i—> which in turn dehnes a G-homogeneous vector bundle 

Efj over P\G. The representation tt^- is the G-representation on the space 
of smooth sections r“(Eo-.,A ) of that bundle. In other words, lives on the 
space of all G“ functions /: G —> with 

f{manx) = a^^~^'^aj{m)f{x) 

for all m G M, a E A, n E N, x E G. The representation nj is dehned 
by nj{y)f{x) = f{xy). Every such / is uniquely determined by its restric¬ 
tion to K which satishes f{mk) = aj{m)f{k), i.e., / is a section of the in¬ 
homogeneous bundle E„. on M\K induced by aj. So the representation space 
can be identihed with = T°^{E„.). Thus a trilinear form T is a distribu¬ 
tion on the vector bundle E„ = E„^ Kl E„^ Kl E^^ over M\K x M\K x M\K. 
Here Kl denotes the outer tensor product. 

For /i, / 2 , /a e G°°{M\K) we write T (/i, / 2 , /s) for the expression 

f 4>{ki, k2, ks) [fi{ki) K f2{k2) H fsiks)] dki d/cs dk^, 

J M\KxM\KxM\K 

where 0 is the kernel of T. 

The group G is called a real hyperbolic group if it is locally isomorphic to 
SO(d, 1) for some d>2. 

On Y = (P\G)^ we consider the G^-homogeneous vector bundle Po-,a given 
by Eg-,A = -^ 0 - 1 ,Ai ^ -^( 72 ,A 2 ® -^ 0 - 3 ,As- Next Y can be viewed as a G-space via 
the diagonal action and so Eg -,a becomes a G-homogeneous line bundle on Y. 

Theorem 2.1 Let Y be the G-space (P\G)^. If there is an open G-orbit in 
Y, then the dimension of the space of invariant trilinear forms on smooth 
principal series representations is less than or equal to 

dim((Ti (g) (J 2 ® 0 - 3 )^°, 

O 

where the sum runs over all open orbits o and Mo is the stabilizer group 
of a point in the orbit o which is chosen so that Mo is a subgroup of M. 
If all induction parameters are imaginary (unitary induction), then there is 
equality. 



INVARIANT TRIPLE PRODUCTS 


6 


In particular, if ni, 712 , are class one representations, then the dimension 
is less than or equal to the number of open orbits in Y. 

There is an open orbit if and only if G is locally isomorphic to a product of 
hyperbolic groups. 

For G = S0(2,1) the number of open orbits is 2, for G = SO{d, 1), d > 3, 
the number is 1 . 


The Proof is based on the following lemma. 


Lemma 2.2 Let G be a Lie group and H a closed subgroup. Let X = G/H 
and let E ^ X be a smooth G-homogeneous vector bundle. Let T be a 
distribution on E, i.e. a continuous linear form on Tf^{E). Suppose that T 
is G-invariant, i.e., T{g.s) = T{s) for every s G Tf^{E). Then T is given 
by a smooth G-invariant section of the dual bundle E*. 

Let (cT, K-) be the representation of H on the fibre E^h and let (a*, I 4 *) be 
its dual. Then the space of all G-invariant distributions on E has dimen¬ 
sion equal to the dimension dim Vfl of H-invariants. So in particular, if a 
is irreducible, this dimension is zero unless a is trivial, in which case the 
dimension is one. 


Proof: The equation T{g.s) = T(s), i.e., g.T = T Wg E G implies X.T = 0 
for every X G 0 r, the real Lie algebra of G. Let [)r be the Lie algebra of H 
and choose a complementary space pR for pR such that Pr = 1)r © pR. Let 
Xi,..., Xn be a basis of V and let 

D = X2 + X2 + --- + X2 G U(0 r). 

We show that D induces an elliptic differential operator on E. By G- 
homogeneity, it suffices to show this at a single point. So let P = exp(pR). 
In a neighbourhood U of the unit in G there are smooth maps h : U ^ H 
and p -. U ^ P such that h{x)p{x) = x for x E U. 

The sections of E can be identified with the smooth maps s : G —W with 
s{hx) = a{h)s{x) ioi h E H and x E G. We can attach to each section s a 
map fs on p with values in W by fsiY) = s(exp(F)). The action of X G pR 
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on the section s is described by 


fxs{Y) 


s(exp(F) exp(tX)) 

(j(/i(exp(F) exp(tX)) s(p(exp(F) exp((tX)). 


Let Ax(Y) 
implies 


^\f-^Q<^{h{exp(Y) expitX)) G End(\4). Then the Leibniz rule 

fxs{Y)=Ax{Y)f,{Y)+Xf,{Y). 


The hrst summand is of order zero and the second of order one. Moreover, 
the second summand at E = 0 coincides with the coordinate-derivative in 
the direction of X. This implies that the leading symbol of D at eH is 
and so D is elliptic. The distributional equation DT = 0 then 
implies that T is given by a smooth section. 


For the second assertion of the lemma recall that a G-invariant section is 
uniquely determined by its restiction to the point eH which must be invariant 
under H. □ 


For the proof of the theorem we will need to investigate the G-orbit structure 
ofY = (P\G)^. First note that since the map Mk i—> Pk is a iL-isomorphism 
from M\K to P\G, the iL-orbit of every y ^ Y contains an element of the 
form {yi,y 2 , 1). Hence the P = MAN-orhit structure of (P\G)^ is the same 
as the G-orbit structure of Y. By the Bruhat decomposition, the P-orbits in 
P\G are parametrized by the Weyl group W = hL(G, H), where the unique 
open orbit is given by PwqP, here Wq is the long element of the Weyl group. 
Note that the P-stabilizer of Pwo G P\G equals AM. This implies that 
the G-orbits in Y of maximal dimension are in bijection to the HM-orbits 
in P\G of maximal dimension via the map PxAM i—> {x,wo,l)-G. Again 
by Bruhat decomposition it follows that the latter are contained in the open 
cell PwqP = PwqN. So the G-orbits of maximal dimension in Y are in 
bijection to the AM-orbits in N of maximal dimension, where AM acts via 
the adjoint action. The exponential map exp: n® —> is an AM-equivariant 
bijection, so we are hnally looking for the AM-orbit structure of the linear 
adjoint action on Ur. 

We will now prove that there is an open orbit if and only if G is locally a 
product of real hyperbolic groups. So suppose that Y contains an open orbit. 
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Then uk contains an open AM-orbit, say AM.Xq. Let be the set of all 
positive restricted roots on a = Lie (A). Decompose Ur into the root spaces 

a€(f>+ 

On each install an M-invariant norm ||.||q,. This is possible since M is 
compact. Consider the map 

-0 : Hr ^ n ^ 

a£<f)+ 

a 

Since the orbit AM.Xq is open, the image iP^AM.Xq) of the orbit must 
contain a nonempty open set. Away from the set {X G Ur : da : ||x||^ = 0} 
the map 'ip can be chosen differentiable. Since the norms are invariant under 
M, one gets a smooth map 

A 

a H->• 'ip{a.Xo), 

whose image contains an open set. This can only happen if the dimension of 
A is at least as big as |0+| and the latter implies that G is locally a product 
of real rank one groups. Now by Araki’s table (j2], pp. 532-534) one knows 
that these real rank one groups must all be hyperbolic, because otherwise 
there would be two different root lengths. 

For the converse direction let G be locally isomorphic to SO{d, 1). We have 
to show that there is an open AM-orbit in ur. This, however, is clear as the 
action of AM on ur is the natural action of X SO{d— 1) on hence 

there are two orbits, the zero orbit and one open orbit. 

We will now show that if there is an open orbit, then there are no invariant 
distributions supported on lower dimensional orbits. For this it suffices to 
consider the case G = SO^{d, 1). For simplicity, we only consider the trivial 
bundle, i.e., functions instead of sections. The general case is similar. The 
orbit structure is as follows. For d = 2 the group M = SO{d — 1) is trivial 
and so there are two open AM-orbits in X = M in this case. If d > 2 there is 
only one open orbit. Since the proof is very similar in the case d = 2 we will 
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now retrict ourselves to the case d > 2. The open orbit [wono,wo, 1], given 
by some uq G N, contains in its closure the orbits [tco, it’o, 1] and [tco, 1,1] 
which in turn contain in their closures the orbit [ 1 , 1 , 1 ]. 

[wono,wo, 1 ] 


U U 

[wo,tao, 1] ko,l,l] 

U U 


[1,1,1] 

Let O = [wono,wo, 1] be the open orbit. Let xi = (wo,wo, 1), then one has 
[tcojiao, 1] = xi.G. Consider A = as a subset of M suitably normalised, 
then one can write 

xi = limxo-am. 

a —^0 

Let T be a G-invariant distribution supported on the closure of the orbit of 
Xi- Since T is G-invariant, it satishes X.T = 0 for everey X G 0 . Hence the 
wave front set WF(T) C T*Y is a G-invariant subset of the normal bundle 
of the manifold xi.G. This implies that T is of order zero along the manifold 
Xi-G. By the G-invariance it follows that T is of the form 


T(/)= [ DCf){x^)dx + R, 

Jxi.G 

where R is supported in [1,1,1]. Further, D is a differential which we can 
assume to be G-equivariant. Then D{f){xi) is of the form 

L)i(m)/(a;o.am)|a=o, 

where Di{m) is a differential operator in the variable a. Since D is G- 
equivariant, we may replace / with “°/ for some oq G A. Since xiGq = xi we 
get that the above is the same as 

L)i(m)/(a;o.aaom)|a=o- 

This implies that Di must be of order zero and so T is of order zero. Re¬ 
stricted to the orbit xi.G = AM\G the distribution T is given by an in¬ 
tegral of the form J^j^^^(l){y)f{xiy)dy. (Note that we use the notation 
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without the dot again.) Invariance implies that 0 is constant. If T is non¬ 
zero, then y i—> f{xiy) must be left invariant under AM, which implies 
Ai + A 2 — A 3 -|- p = 0. But then the distribution does not extend to all of Y 
as the integral will not converge if / is non-zero in (1,1,1). So T must be 
zero. This shows that any invariant distribution which is zero on the open 
orbit also vanishes on xi.G. The remaining orbits are dealt with in a similar 
fashion. To prove the Theorem, it remains to show the existence of invariant 
distributions. For this we change our point of view and consider sections 
of Lx no longer as functions on Y, but as functions on whith values in 
Va = ® 14-2 ® ^ 0-3 which spit out on the left. We induce this 

in the notation by writing f{xoy) instead of f{xQ.y). On a given orbit of 
maximal dimension, there is a standard invariant distribution which, by the 
lemma, is unique up to scalars and given by 

%tf = “ (y f\xoy) dy^ , 

where a is a linear functional on the space of Mo-invariants. In order to show 
that this extends to a distribution on Y, we need to show that the dehning 
integral converges for all / G This integral equals 


f{xoy)dy = / f{wonoy,Woy,y)dy 


'G 


IG 


a{wonoyY^^^a{woyy^^^a{yy^^^ f{k{xoy)) dy. 


'G 


Since / is bounded on K^, it suffices to show the following lemma. 
Lemma 2.3 Let G = SO{d, 1)° and let ko = k{wono). Then 


a{wonoyya{woyya{yydy < 00 . 


'G 


Conjecture 2.4 The assertion of the lemma should hold for any semisimple 
group G with finite center and Uq E N generic. 


The conjecture would imply that if G is not locally a product of hyperbolic 
groups, then the space of invariant trilinear forms on principal series repre¬ 
sentations is inhnite dimensional. 
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Proof of Lemma I2.8t Replace the integral over G by an integral over ANK 
using the Iwasawa decomposition. Since gJ^xk) = g{x) for x G G and k E 
the R'-factor is irrelevant and we have to show that 

/ g{kQanYg{wQanyda dn < oo. 

J AN 

Now write wqUo = a'n'ko. Then koan = {a'n')~^wonoan and therefore 
g{koan) = g{wonoan), so it suffices to show 


g{wonoanyg{woan)^a^ dadn < 


'AN 


OO. 


Next note that woa = a and so we have aiw^any = a ^aiw^ny as well 
as giwQTiQany = a~Pg{wonQn)'^, where ng = a“^noa. We need to show 


I AN 


giwQrynya{wQnya ^ da dn 


< 


oo. 


This is the point where we have to make things more concrete. Let J be 
the diagonal (d + 1) x (d + l)-matrix with diagonal entries —1). 

Then SO(d, 1) is the group of real matrices g with g^Jg = J. Writing g = 

^ with A G Matrf(M), this amounts to 

A^A - Pc = 1 
A^b-Cd = 0 
d^-Rb = 1 


f A b 
y c d 


The connected component SO(d, 1)° consists of all matrices g as above with 
d > 0. The maximal compact subgroup K can be chosen to be ' 


SO(d-l) 


and M as 


0 

0 . Further, we can choose A and N as follows, 





A = 

( « 

/3 


1V 

a 


/3 : a > 0, = 1 > , 


N = < nix) = 



—X 

X 

\ 

\ 

X* 

1 hP 

2 

|x|V2 


: X M 

X* 

— X ^/2 

1 + ^ 
^ 2 

) 

1 
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where we have written \x\‘^ 
of A is generated hj H = 


= x\ + x\ + ■ ■ ■ + Note that the Lie algebra 

\ 

0 1. One derives an explicit formula 


for the AiViL-decomposition. In particular, if g 


A b 
c d 


and a{g) = 



a{g)P = (a + 


( {d + hd) \ — 

\l + hl + --- + hlJ 



So that with n = n{x) for a; G ^ we have 



We choose no = n(l, 0,..., 0) and get with a = exp(tH), 


gJgxonQnY 



d-1 

2 
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With n = d — 1 and a ^ = e 2 * our assertion boils down to 

[ [ + x\‘^)~^dtdx < 00 . 

Jr" Jr 

Consider first the case n = 1 and the integral over a; < 0: 

e“5*(l + + (e“* + xy)~^dxdt 





e 2^(1+ a;^) 2(1 +(x — e ^dxdt 


e 2 ‘(l + (a; + e ) ) 2 (i + (a;) ) ‘^dxdt. 


Thus it suffices to show the convergence of 



e 2*(i + a;^) 2 (i-|-( 3 ; + e ^dtdx. 


Setting y = e 'we see that this integral equals 


^>00 /*oo 


y + + (x + ^^‘^dydx 


'0 Jo 

poo poo 


{y — x) + + ^^‘^dydx 


Jo J X 

Since the mapping 


(y — x) + ^'^^(1 + 1/^) ^^“^dy 

J X 

is continuous, the integral over 0 < a; < 1 converges. It remains to show the 
convergence of 


^00 poo 


{y — x) ^'^^(1 + a;^) ^^‘^{l + y"^) ^^‘^dydx 


' 1 J X 


r*oo poo 


xV\v - + a; 2 )-V 2 (i + v^x^Y^/Hvdx 


'1 Ji 


f'OO poo 


< 3 - 1/2 y J a;i/ 2 (n-l)-i/ 2 (l + a; 2 )-i(l + n 2 )-i/ 2 dt;da; < 00 . 
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here we have used the substitution y = vx and the fact that for a, 6 > 1 one 
has (1 + a)(l + b) < 3(1 + ab). 

Now for the case n > 1. Using polar co-ordinates we compute 

[ [ [ + + \Xr\‘^)~^^‘^dtdxdXr 


= c 


g-in/2^n-2(^ 4- + r^)-”/2(i+)e-* -h xf + r'^Y^I'^dt dx dXr 


As above we can restrict to the case a; > 1. We get 


oo roc poo 


0 -'0 -'0 
poo poo poo 


y 2 ‘^{1 + x‘^ + Y) {1 + {x + yY + Y) "'^‘^dydxdr 
{y — x) 2“V"'“^(1 + + y^ + Y)~^^‘^dydxdr, 


0 JO Jx 


which equals 


OO poo poo 


(n — 1)2 "^{l + x^ + Y) ”'^^(1 + + r^) "'^‘^dydxdr. 


0 JO Jl 


As above, it suffices to restrict the integration to the domain a; > 1. So one 
considers 


OO poo poo 


(n-l)2 ^(l + a;^ + r^) + v'^x'^ + Y) '^^‘^dydxdr. 


Choose 0 < e < 1/2 and write 

(1 + a;^ + = {1 + x'^ + + x"^ + Y)~^ 

< {1 + + x‘^)-L 

So our integral is less than or equal to 

/ OO 

+ Yy-'^/^dr 


times 


(n —1)2 ^a;’"/^(l + a;^) ^(1 + n^a;^) "'^‘^dvdx 


(^;_l)f-V/2(l+a;2)-e(i^^2)-n/2(i^^2)-n/2^^^^ < CX). 
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3 An explicit formula 

Let d > 2 and G = SO{d, 1)° if d > 2. For d = 2 let G be the double 
cover Sb(2,1)° ^ PGL 2 (M). Then K = SO(d), M = SO(d - 1) for d > 2. 
For d = 2 we have K = 0(2) and M = 'Ll2% and in all cases we have 
M\K = the d — 1 dimensional sphere. For each Ago* let e\ be the 

class one vector in the associated principal series representation tta given by 
e\(ank) = Let A,/i, z/ G o* be imaginary. Let %t be the invariant 

distribution on considered in the last section. We are interested in 

the growth of d^t(eA, e^, e,y) as a function in A. First note that the Killing 
form induces a norm |.| on a*. 

We write 7^t(A, /i, v) for 7^t(eA, e^, Gy) and identifying or to M via A i—> A(ido) 
we consider as a function on (iR)^. 

In this section we will prove the following theorem. 

Theorem 3.1 For A,/i, z/ imaginary, %t(\,ia,i') equals a positive eonstant 
times 


p ^ 2(A+//—^ jn ^2(A—^+z/)+n^ jn —A+^+z/)+n^ jn ^ 2(A+/i+i^)+n ^ 

F (^) F (^) F (^) ’ 

where n = d—1. So in particular, for fixed imaginary p and u. Then, as |A| 
tends to infinity, while A is imaginary, we have the asymptotic 

|Tst(A,/i,z/)| = cexp (-||A|) \\\^~^ (^1 + 0 > 

for some constant c > 0. 

Proof: The asymptotic formula follows from the explicit expression and the 
well known asymptotical formula, 

|F(a + ft)| = v^exp (-||f|) l^r"^ (^1 + 0 

as |t| oo, where the real part a is hxed. 
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Now for the proof of the hrst assertion. The permutation group S '3 in three 
letters acts on by permuting the co-ordinates. We claim that Tgt is 

invariant under that action. So let a E S 3 and let / = /i ® /2 ® /s, then 

= [ f{(^~^{xo-y))dy 

Jg 

= [ f{(^~\xo).y)dy 

Jg 

Since the open orbit is unique, there is y^ E G with a~^{xo) = XQ.y^, and so 

= [ f{xo.y„y)dy 

Jg 

= [ f{xQ.y)dy = %t{f). 

Jg 

So in particular z/) is invariant under permutations of (A,/i, u). 

We have 


%t{ex,e^,e^) = / a{wonoy)^^^a{woyY^^a{yy^f’dy. 

Jg 


Replace the integral over G by an integral over ANK using the Iwasawa 
decomposition. Since a{xk) = qJ^x) ioi x E G and k E K, the if-factor is 
irrelevant and we have to compute 



a{wonoan)^~^^a{woan)^^^dadn. 


Note that Woa = a ^Wq and so we have a(woan)^ = a ^a(won) as well as 
a(wonoan) = a~^a(wonQn), where Uq = a~^noa. So the integral equals 


u-X-n-p 


IAN 


a{wonQn)^~^^a{won)^~^^ da dn. 


Using the explicit approach of the last section we see that 7^t(A,/i, ix) equals 

+ |e-% + x\Y^-^{l + \x\Y^-^ dxdt, 
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where vi = (1, 0,..., 0). 

Set n = h — 1 = 1, 2, 3,... and use polar co-ordinates to compute for n > 1, 
that 





'(1 + 1 


e ‘ui-f-xp) 2 (l-t-|a;p) ^ ^dxdt 


equals (n — l)vol(-B„_i) times 

j-g-t _(_^2 _|_ dxdt. 

Dehne /n(a, b, c) to be 

+ (e-* -t- xf + r2)-“(l -l- + r^y^drdxdt, 

and /i(a, b, c) to be equal to 





^^(l-|-(e “(1-l-a;^) ’^drdxdt. 


Then, for all n G N, 


u) 


r f. n n n\ 
Cnin ^ 2' ^ ^ 2' ^ ^ 2 ) ' 


where Ci = 1 and Cn = {n — l)vol(-B„_i) for n > 1. So in particular, R is 
invariant under permutations of the arguments. 


Note that 


equals 


^ [(1 + (e-‘ -t- xf + r2)-“(l + a;2 + 

-2ar(l + (e-* + xf + + a:' + 

-2br{l + (e-' + xf + r2)-“(l + + rY'~\ 


and ^ 


d -r^ ^ 
dr n—1 


So, integrating by parts, for n > 1 we compute 


In{a,b,c) = 


- -In+2{<^ -|- 1, 6, C -|- 1) H- -In+2{(1, & S" 1, C + 1). (1) 

n—1 n—1 
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To get a similar result for Ji note that 


equals 

-(1 + (e-* + xf + r2)-“(l + 

^ [(1 + (e“‘ + x)^ + (ir. 

This implies 



/i(a, b, c) — 20 / 3(0 + 1 , 6 , c + 1 ) + 26 / 3 ( 0 , 6 + 1 , c + 1 ). ( 2 ) 

Let /(i(o, b, c) be the same as /„(«, b, c) except that there is a factor x in the 
integrand, i.e., 

r"'-^e*(‘^-“-^)(l + (e“* + x)^ + r^)““(l + x^ + x dr dx dt. 

Replacing x with —x first and then with x + e“* yields 

= -/(,(/, o, c)-/„(/, o,c-1). (3) 

The last equation also holds for n = 1. Integration by parts gives 

Jr 



2a 

a + b — c 


gi(c-a-fe)^^ + (e-* + xY + r2)-“-i(e-2* + xe”*) dt. 


So, 

/„(o, 6 , c) = ——-(/„(o + 1 , 6 , c - 1 ) + /(,(o + 1 , b, c)). 

a T b — c 

Using 0 this yields 


or 


2a 


In(a, b, c) = - -T^{b, o + 1 , c), 

c — a — b 


PI N z-x-y+l 

4(u y, z) = —^— in{y - 1 , X, z). 


( 4 ) 
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Plugging this into dBD one gets 


6 + 1 


26-2 


4(6-1, a, c) = 


(X b 


2a-2 


-In{a - 1, 6, c) - 4(6, a, c - 1). 


Replacing (a, 6, c) with (a + l,6+l,c+l) this gives 
c — a — b 


26 

a + 6 — c 


2 a 

or, using the invariance of 4, 
c — b — a 


In{b, a + 1, c + 1) 

4(^5 6 + 1, c + 1) — 4(6 + 1, a + 1, c). 


2 ab 

and hence 


[aIn{oi + 1, 6, c + 1) + bln{a, 6 + 1, c + 1)) — — In{,o, + 1, 6 + 1, c). 


®4+2(® + 1,6, c + 1) + 64+2(®) 6 + 1, c + 1) — 
Using ((T)) and m one gets for n > 1, 


2 ab 


a + b — c 


4+2(0 + 1 , 6 + 1 , c). 


4a6 1 / 7 N 

4(0, 6, c) = ——- r4+2(0 + 1, 6 + 1, c), 

a + 6 — cn — 1 


and 

^ab 

4(0, 6, c) = ——-4(a + 1, 6 + 1, c). 

a + 0 — c 

Finally we arrive at 

4+2(0 + 1,6 + 1, c + 1) = ^ - l)-^n(o, 6, c + 1) 

for 77, > 1 and 


4(0 + 1,6 + 1, c + 1) — --— - -4(0,6, c + 1). 

4a6 

These formulae amount to 




A + /i — u ^ — 1 

(2A + n)(2/i + n) 


A 1 ), 


( 5 ) 

( 6 ) 
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where dn = [n — l)cn+2/cn if n > 1 and di = C3. A calculation using the 
functional equation of the Gamma-function, r(2; -|- 1 ) = zT{z), shows that 
the claim of the lemma for n implies the same claim for n -|- 2 . Note that 
the formula A .5 in P] implies the lemma for n = 1, where one has to take 
into account that in P] a different normalisation is used. To get our formula 
from theirs, one has to replace \j by — 2 Aj in pQ. To hnish the proof of the 
lemma it remains to show the claim for n = 2 . 

To achieve this, we proceed in a fashion similar to First note that the 
group G = SL2(C) is a double cover of SO( 3 , 1 )°, so we might as well use this 
group for the computation. For A G C let Vx denote the space of all smooth 
functions / on with f{az,aw) = f {z, w) for every a G C^. Then 

G = SL2(C) acts on the space Vx via 7 ix{g)f{z,w) = f{{z,w)g). This is 
the principal series representation with parameter A. For A = 1 there is a 
G-invariant continuous linear functional L: Vi —C, which is unique up to 
scalars and is given by 

Hf) = f f{x)dx, 

Js3 

where the integral runs over the standard sphere C with the volume 
element induced by the standard metric on Since equals ( 0 , l)K, 

where K = SU( 2 ) is the maximal compact subgroup of G, the theory of in¬ 
duced representations shows that this functional is indeed invariant under G 
and is unique with that property. If A is imaginary, then the representation tta 
is pre-unitary, the inner product being given by (/, S') = L{fg). To compute 
^t(Ai, -^2, A3), we will describe this functional on the space Vx^ ® Va2 G) Vx^. 
Let oj: —*■ C be given by uj{a, b) = 0162 — a-2bi the bilinear G-invariant on 
C^. Let A'ai,a2,A3('i^i,"^ 2,1^3) be the invariant kernel 

V2) I I (A1-A2+A3-I) I (Ai+A2-A3-1)_ 

This kernel is invariant under the diagonal action of G and is homogeneous 
of degree 2 (Aj — 1 ) with respect to the variable Vj. From this point the 
computations run like in P] A .4 and A. 5 . The theorem follows. □ 
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A Automorphic coefficients 

Let r C G be a uniform lattice and consider the right regular representation 
of G on L^(r\G), which decomposes as a direct sum, 

r"(r\G) = 0i"(r\G)(7r), 

ttGG 

where the isotypic component L^(r\G)(7r) is zero for tt outside a countable 
set of TT G G and is always isomorphic to a sum of hnitely many copies of tt. 
By the Sobolev lemma one has L'^(r\G)°° C G°°(r\G). A function cp is called 
pure, if G L^(r\G)(7r) for some tt G G. For three pure smooth (pi,(p2,V^3 
the integral (pi(x)(p2(x)(p3(x) dx exists. Moreover, £x 7ri,7r2,7r3 G G and 
§x G-equivariant embeddings aj: > L^(r\G), then 

'T^nt{Vi,V2,V^) = / (Ti{Vi){x)(T2{v2){x)(T‘i{v^){x)dx 

Jv\G 

dehnes a G-invariant continuous trilinear form on x x V^. Let 
L^(r\G)^ be the space of all iF-invariant vectors in L^(r\G). Then there 
is an orthonormal basis ((pi)ieN of pure vectors in L^(r\G)'^. For each i let 
Li be the G-stable closed subspace of L^(F\G) generated by (pj. Then the 
spaces Li are mutually orthogonal. 

For hxed pure normalized ip, ip' G L^(F\G)^ we are interested in the growth 
of the sequence 

Ci = T'mi^i^ipi, ip, ip'). 

Let {Xi,fi,i>) be the induction parameters of the representations given by 
{ipi, ip, ip'). First note that the numbers c, decay exponentially in |Aj|. So we 
dehne the renormalized sequence 

bi |Q|^exp( 7 r|Aj|). 

In jTj it is hown that for d = 2 one has 

6 , < GlogT, 

|A,;|<T 
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and in |3], Theorem 7.6, that for d > 3 , 

|Ai|<T 

We are going to reprove this resnlt for d > 4 . 

Theorem A.l Let d > 4. There is C > 0 such that for every T > 0 , 

|Ai|<r 

Proof: By the nniqneness of trilinear forms there exists a constant a* snch 
that 

This constant depends on the embeddings of tt;^, vr^,, tt^ into L^(r\G), bnt 
we can normalize these embeddings by insisting that the standard class one 
vectors ca, e^, are mapped to (^j, (^, y)'. Thns the nnmber Oi only depends 
on ipi, tp, ip'. 

Fix two smooth pre-nnitary G-representations V, V' and let L, L' be their 
Hilbert completions. Consider the nnitary representation of G x G on L ® L' 
and let E = {L® L')°° be its smooth part. Denote by 'H{E) the real vector 
space of all Hermitian forms on E and by TD{E) the cone of non-negative 
forms. 

Let VF be a smooth pre-nnitary admissible representation of G. A G-invariant 
fnnctional T\W®V®V'^C dehnes a G-eqnivariant linear map Ir'- V ® 
V —> W* which extends to a G-map Ir'- E ^ W, where W is the space 
W with complex conjngate linear strnctnre. The Hermitian form on W 
indnces a form on W. Dehne a Hermitian form Hq- on E by Hr = 
i.e., for u,v & E one has Hr{u,v) = Hf^{lr{u),lr{v)). Finally, snppose 
that W = Va, F, are all principal series representations and let %t be the 
standard trilinear form then we write Hf for the Hermitian form induced on 
E. We also assume that V\^V,V' are cuspidal, i.e., that hxed embeddings 
into are given. Then the cuspidal trilinear form T induces a Hermitian 
form on E. 


INVARIANT TRIPLE PRODUCTS 


23 


Next consider the space L‘^’°°{r\G x r\G). Let denote the Hermitian 
form on L^’°°(r\G x r\G) given by restriction to the diagonal, i.e., 


Ha {u,v) = / u{x,x)v{x,x) dx. 

Jr\G 

Let U he a Hilbert space with Hermitian form H. For a closed snbspace L 
of U let Ptl denote the orthogonal projection from U to L and let Hl he 
the Hermitian form on U given by 

Hl{u,v) = H{PrL{u),PrL{v)). 

The map L i—> Hl is additive and monotonic, i.e., Hl+l' = Hl + Hl' if L 
and L' are orthogonal and Hl < Hl' if L C L'. 

Lemma A. 2 On the space E one has 

i 

Proof: By the nniqneness it follows H^'^ = and since all the spaces 

Li are orthogonal one also dednces Yli H^^ < Ha- □ 

The gronp G x G acts on L^’°°(r\G x r\G) and thns on the real vector 
space of Hermitian forms on L^’°°(r\G x r\G). By integration, this action 
extends to a representation of the convolntion algebra C“(G x G). Note that 
a non-negative fnnction h G G^ {G x G) will preserve to cone of non-negative 
Hermitian forms. 

Lemma A. 3 Let h G G^(G x G) be non-negative. Then there exists G > 0 
such that 

h.HA < GH^-ati 

where LTaut is the L"^-Hermitian form on L^'°°(T\G x r\G). 


Proof: The form h.HA is given by integration over a smooth measnre on 
r\G X r\G. Being smooth, this measure is bounded by a multiple of the 
invariant measure. □ 
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A positive functional on 7i(-E) is an additive map p: 1HA{E) —> M>o U {cxo}. 
For example, every vector u E E gives rise to a positive fnnctional defined 
as 

5u{H) = H{u,u). 

Every positive fnnctional is monotonic and homogeneous, i.e., 

p{H) < p{H') if H<H', 

and 

p{tH) = tp{H) for t > 0. 

Let p be a positive functional and let hp{\) = p{Hf). Then Lemma lA.21 
implies 

Y,hp{\f)\a,\^ < p{H^). 


Proposition A.4 There are constants Tq, C > 0 such that for every T >Tq 
there is a positive functional pr on 74(E) with 

pAHa) < 

and with hx = hp^ one has hT{X) > ^ for every |A| < 2T. 

Proof: Let be a relatively compact open subset of G with g E Di ^ g~^ G 
D. Assume further that D is invariant under right and left translations by 
elements of K. Let h E C'^{G x G) such that h is constantly one on D x D 
and such that h is invariant under left and right translates by elements of K. 
the space E can be identified with G°°{M\K x M\K). Fix a norm H-H on 
the Lie algebra of AN. 

Note that the map 

Ip: AxN xM P\G X P\G 

(a, n, m) i—^ {woanm, wonoanm) 

is a diffeomorphism onto an open subset of {P\G)‘^ = {M\K)‘^. Let F = 
-0(1 X 1 X M) = the M-orbit of the point {wo,wono). Let Gi,T > 0 be large 
and set 
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Let ut be a nonnegative smooth function on {M\K)‘^ with support in S such 
that 

j udxdy = 1, j \u\^dxdy < , 

It is possible to construct such a function since the codimension of F equals 
dim(M\iL) + 1. With these data dehne a positive functional px on TiiE) by 

Pt{H) = duAh.H). 

Then one has 

= Puih.H^) < C'p^{H,t) < CT^rniM\K)+i^ 

Recall the standard trilinear form %tonV\OV®V' and the corresponding 
linear map V ®V' ^ Vx. We identify the space Vx with C°°{M\K) 
and let denote the point Ml G M\K. The Dirac measure 5z at is a 
continuous linear functional on C°°{M\K). We get an induced Hermitian 
form Hz on E dehned by 

Hziu^v) = l%,{u)i.z)l%tiT){.z). 

Lemma A.5 On E one has 

Hf = [ {k,k).Hzdk. 

Jk 

Proof: This follows from the fact that the invariant Hermitian form on Vx 
is equal to Jp.7r{k)Hzdk, where Hz is the Hermitian form on Vx given by 
Hz{u,v) = u{z)v{z). □ 

Since the test function h was assumed to be A x A-invariant it follows that 

hriX) = Pt W) = Suih.Hz). 

So in order to establish a lower bound for pxiHf') it suffices to give a lower 
bound for Su{x.Hz) = \ {n{x)fz,u) p for x E D, where fz is the linear func¬ 
tional on E given by fziu) = Sz{l%t{u)). 
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Lemma A.6 There is C > 0 such that for Tq large enough there exists an 
open, non-empty subset Dq C D such that for T > Tq, |A| <T, and x E Dq 
we have \ {7i{x)fz,UT) I > 


Proof: Note first that with x = ank E G we have 

{7l{x)fz,UT) = [ UT{k2,ks)4>{k,kr^kr’^)dk2dk3. 

J(M\KP 

We next derive an explicit formula for 0 on the open orbit. Recall that 


TM) 



f{y,W)y,wonoy) dy 


f{ank, woank, woUoank) da dn dk, 


J ANK 

and that the last integrand equals 

afwoanY^'^ a{wQnoany^'^ f {k, k{woan)k, k{wonoan)k) da dn dk. 
We deduce that 


<j){k, k{woan)k, k{wonoan)k) = a^~^^a{woanY^'^a{wonoany~^^W{an), 

where W is the Radon-Nykodim derivative of dadndk against the measure 
on {M\Ky. In particular, this function is positive and smooth and does not 
depend on A. It follows that the functional 7i{x)fz is given by integration 
against the function 

7r{ank)fz{woa'n'm',wonoa'n'm') = (f){k,woaa'n"m'k,wonoaa'n"m'k), 

where n" = a~^n'am'n{m')~^. Note that Mk = Mm'k and so the above 
equals 

{aa'y^'^gfwQaa'n"y~^'^gfwQnoaa'n"y~^^W{aa n"). 

Note in particular, that the function 7r{x)fz is invariant under M. 

The set 5* is a tubular neighbourhood of the compact set F. We have natural 
local coordinates with F-directions, which can be identihed with open parts 
of M-orbits and F“*“-directions, which can be viewed as AA-orbits. The 
above formula shows that the gradient of K{x)fz is zero along the M-orbits. 
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Along the AiV-orbits this gradient is bonnded by a constant times T, if we 
leave fi and u hxed and restrict to |A| < T. Let Co > 0 be large and let 
Do CZ D be the snbset of elements cc ^ D snch that L^o* 'f his set 

is open and nonempty if Cq is large enpongh and it does not depend on A. 

T;emma lA.6l implies Proposition IA.4I and the latter implies Theorem lA.II as 
follows. 

Consider the ineqnality < pt{H/:^). By Proposition I A. 41 

the right hand side is bonnded by ( 72 " 2 dim{M\K) since dimM\iL = 
dim P\G = dim = d — 1 we get 

^ cr". 

i 

Restricting the snm to those i with |Aj| < 2T and using the second assertion 
of Proposition IA.4I we get 


\ai\^ < CT^. 

|Ai|<T 


According to Theorem 14.11 we have ~ |Ojp and so 

5 ^ < CT’^. 

|Ai|<T 


This implies the theorem. 


□ 
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